{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Introduction to Docker\n",
    "\n",
    "**Learning Objectives**\n",
    "  * Build and run Docker containers\n",
    "  * Pull Docker images from Docker Hub and Google Container Registry\n",
    "  * Push Docker images to Google Container Registry"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Overview\n",
    "\n",
    "Docker is an open platform for developing, shipping, and running applications. With Docker, you can separate your applications from your infrastructure and treat your infrastructure like a managed application. Docker helps you ship code faster, test faster, deploy faster, and shorten the cycle between writing code and running code.\n",
    "\n",
    "Docker does this by combining kernel containerization features with workflows and tooling that helps you manage and deploy your applications.\n",
    "\n",
    "Docker containers can be directly used in Kubernetes, which allows them to be run in the Kubernetes Engine with ease. After learning the essentials of Docker, you will have the skillset to start developing Kubernetes and containerized applications."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Basic Docker commands"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "See what docker images you have. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!docker images"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "If this is the first time working with docker you won't have any repositories listed. \n",
    "\n",
    "**Note**. If you are running this in an AI Notebook, then you should see a single image `gcr.io/inverting-proxy/agent`. This is the container that is currently running the AI Notebook. \n",
    "\n",
    "Let's use `docker run` to pull a docker image called `hello-world` from the public registry. The docker daemon will search for the `hello-world` image, if it doesn't find the image locally, it pulls the image from a public registry called Docker Hub, creates a container from that image, and runs the container for you."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!docker run hello-world"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now when we look at our docker images we should see `hello-world` there as well."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!docker images"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This is the image pulled from the Docker Hub public registry. The Image ID is in `SHA256` hash format—this field specifies the Docker image that's been provisioned. When the docker daemon can't find an image locally, it will by default search the public registry for the image. Let's run the container again:"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now, if we want to run `docker run hello-world` again, it won't have to download from the container registry."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To see all docker containers running, use `docker ps`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!docker ps"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "There are no running containers. **Note. If you are running this in at AI Notebook, you'll see one container running.**\n",
    "\n",
    "The `hello-world` containers you ran previously already exited. In order to see all containers, including ones that have finished executing, run docker `ps -a`:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!docker ps -a"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This shows you the Container ID, a UUID generated by Docker to identify the container, and more metadata about the run. The container Names are also randomly generated but can be specified with docker run --name [container-name] hello-world."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Build a Docker container"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's build a Docker image that's based on a simple node application. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Open a new text file and write the following. Save the file in a folder called `dockerfiles` and name the file `intro.docker`"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "```bash\n",
    "# Use an official Node runtime as the parent image\n",
    "FROM node:6\n",
    "\n",
    "# Set the working directory in the container to /app\n",
    "WORKDIR /app\n",
    "\n",
    "# Copy the current directory contents into the container at /app\n",
    "ADD . /app\n",
    "\n",
    "# Make the container's port 80 available to the outside world\n",
    "EXPOSE 80\n",
    "\n",
    "# Run app.js using node when the container launches\n",
    "CMD [\"node\", \"./src/app.js\"]\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This file instructs the Docker daemon on how to build your image.\n",
    "\n",
    "The initial line specifies the base parent image, which in this case is the official Docker image for node version 6.\n",
    "In the second, we set the working (current) directory of the container.\n",
    "In the third, we add the current directory's contents (indicated by the \".\" ) into the container.\n",
    "Then we expose the container's port so it can accept connections on that port and finally run the node command to start the application.\n",
    "\n",
    "Check out the other [Docker command references](https://docs.docker.com/engine/reference/builder/#known-issues-run) to understand what each line does."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We're going to use this Docker container to run a simple node.js app. Have a look at `app.js`. This is a simple HTTP server that listens on port 80 and returns \"Hello World.\"\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now let's build the image. Note again the \"`.`\", which means current directory so you need to run this command from within the directory that has the Dockerfile.\n",
    "\n",
    "The `-t` is to name and tag an image with the `name:tag` syntax. The name of the image is `node-app` and the tag is `0.1`. The tag is highly recommended when building Docker images. If you don't specify a tag, the tag will default to latest and it becomes more difficult to distinguish newer images from older ones. Also notice how each line in the Dockerfile above results in intermediate container layers as the image is built."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!docker build -t node-app:0.1 -f dockerfiles/intro.docker ."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's check that the image has been created correctly. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!docker images"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You should see a `node-app` repository that was created only seconds ago. \n",
    "\n",
    "Notice `node` is the base image and `node-app` is the image you built. You can't remove `node` without removing `node-app` first. The size of the image is relatively small compared to VMs. Other versions of the node image such as `node:slim` and `node:alpine` can give you even smaller images for easier portability. The topic of slimming down container sizes is further explored in Advanced Topics. You can view all versions in the official repository here.\n",
    "\n",
    "Note, you can remove an image from your docker images using `docker rmi [repository]:[tag]`."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Run a Docker container\n",
    "\n",
    "Now we'll run the container based on the image you built above using the `docker run` command. The `--name` flag allows you to name the container if you like. And `-p` instructs Docker to map the host's port 4000 to the container's port 80. This allows you to reach the server at http://localhost:4000. Without port mapping, you would not be able to reach the container at localhost."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!docker ps -a"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!docker run -p 4000:80 --name my-app node-app:0.1"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To test out the server, open a terminal window and type the following command:\n",
    "\n",
    "```bash\n",
    "curl http://localhost:4000\n",
    "```\n",
    "\n",
    "You should see the server respond with `Hello World`"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The container will run as long as the initial terminal is running. If you want to stop the container, run the following command in the terminal to stop and remove the container:\n",
    "\n",
    "```bash\n",
    "docker stop my-app && docker rm my-app\n",
    "```\n",
    "After a few moments the container will stop. You should notice the cell above will complete execution.\n",
    "\n",
    "#### Running the container in the background\n",
    "If you want to the container to run in the background (not tied to the terminal's session), you need to specify the `-d` flag.\n",
    "Now run the following command to start the container in the background"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!docker run -p 4000:80 --name my-app -d node-app:0.1"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Your container is now running in the background. You can check the status of your running container using `docker ps`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!docker ps"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Notice the container is running in the output of docker ps. You can look at the logs by executing `docker logs [container_id]`. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Note, your container id will be different\n",
    "!docker logs b9d5fd6b8e33"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You should see \n",
    "```bash\n",
    "Server running at http://0.0.0.0:80/\n",
    "```\n",
    "If you want to follow the log's output as the container is running, use the `-f` option."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Modify & Publish\n",
    "\n",
    "Let's modify the application and push it to your Google Cloud Repository (gcr). After that you'll remove all local containers and images to simulate a fresh environment, and then pull and run your containers from gcr. This will demonstrate the portability of Docker containers.\n",
    "\n",
    "### Edit `app.js`\n",
    "Open the file `./src/app.js` with the text editor and replace \"Hello World\" with another string. Then build this new image. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!docker build -t node-app:0.2 -f dockerfiles/intro.docker ."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Notice in `Step 2` of the output we are using an existing cache layer. From `Step 3` and on, the layers are modified because we made a change in `app.js`.\n",
    "\n",
    "Run another container with the new image version. Notice how we map the host's port 8000 instead of 80. We can't use host port 4000 because it's already in use. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!docker run -p 8000:80 --name my-app-2 -d node-app:0.2"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You can check that both container are running using `docker ps`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!docker ps"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "And let's test boht containers using `curl` as before:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!curl http://localhost:8000"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!curl http://localhost:4000"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Recall, to stop a container running, you can execute the following command either in a terminal or (because they are running in the background) in a cell in this notebook. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Publish to gcr\n",
    "\n",
    "Now you're going to push your image to the Google Container Registry (gcr). To push images to your private registry hosted by gcr, you need to tag the images with a registry name. The format is `[hostname]/[project-id]/[image]:[tag]`.\n",
    "\n",
    "For gcr:\n",
    "\n",
    "  * `[hostname]`= gcr.io\n",
    "  * `[project-id]`= your project's ID\n",
    "  * `[image]`= your image name\n",
    "  * `[tag]`= any string tag of your choice. If unspecified, it defaults to \"latest\"."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import os\n",
    "\n",
    "PROJECT_ID = \"your-gcp-project-here\" # REPLACE WITH YOUR PROJECT NAME\n",
    "os.environ[\"PROJECT_ID\"] = PROJECT_ID"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's tag `node-app:0.2`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!docker images"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%%bash\n",
    "docker tag node-app:0.2 gcr.io/${PROJECT_ID}/node-app:0.2"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now when we list our docker images we should see this newly tagged repository."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!docker images"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Next, let's push this image to gcr."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%%bash\n",
    "docker push gcr.io/${PROJECT_ID}/node-app:0.2"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Check that the image exists in `gcr` by visiting the image registry Cloud Console. You can navigate via the console to `Navigation menu > Container Registry` or visit the url from the cell below:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%%bash\n",
    "echo \"http://gcr.io/${PROJECT_ID}/node-app\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Test the published gcr image\n",
    "\n",
    "Let's test this image. You could start a new VM, ssh into that VM, and install gcloud. For simplicity, we'll just remove all containers and images to simulate a fresh environment.\n",
    "\n",
    "First, stop and remove all containers using `docker stop` and `docker rm`. **Be careful not to stop the container running this AI Notebook!**."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!docker stop my-app && docker rm my-app"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!docker stop my-app-2 && docker rm my-app-2"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now remove the docker images you've created above using `docker rmi`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!docker images"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%%bash\n",
    "docker rmi node-app:0.2\n",
    "docker rmi gcr.io/${PROJECT_ID}/node-app:0.2\n",
    "docker rmi node-app:0.1\n",
    "docker rmi node:6 \n",
    "docker rmi -f hello-world:latest"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Confirm all images are removed with `docker images`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!docker images"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "At this point you should have a pseudo-fresh environment. Now, pull the image and run it."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%%bash\n",
    "docker pull gcr.io/${PROJECT_ID}/node-app:0.2\n",
    "docker run -p 4000:80 -d gcr.io/${PROJECT_ID}/node-app:0.2"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You can check that it's running as expected using before:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!curl http://localhost:4000"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Copyright 2020 Google LLC Licensed under the Apache License, Version 2.0 (the \"License\"); you may not use this file except in compliance with the License. You may obtain a copy of the License at https://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an \"AS IS\" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License."
   ]
  }
 ],
 "metadata": {
  "environment": {
   "name": "tf2-gpu.2-1.m59",
   "type": "gcloud",
   "uri": "gcr.io/deeplearning-platform-release/tf2-gpu.2-1:m59"
  },
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.8"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}
